Frame rate detection method and frame rate conversion method

ABSTRACT

A method of detecting a frame rate of a source, including: receiving frames of a source provided at an input frame rate; providing the frames to a window having a predetermined length to detect the number of original frames included within the window having the length; and multiplying the input frame rate divided by the length and the number of original frames.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from and the benefit of Korean PatentApplications No. 10-2017-0180662, filed on Dec. 27, 2017 and10-2017-0180665, filed on Dec. 27, 2017, which are all herebyincorporated by reference in their entirety.

BACKGROUND 1. Field of the Invention

The present invention relates to a frame rate detection method and aframe rate conversion method.

2. Discussion of Related Art

Frame rate conversion (FRC) or motion compensated frame interpolation(MCFI) is an important function of a current display device including adigital TV, and has an objective of removing motion blur and motionjudder. The motion blur is caused by a low source frame speed or aproperty of sampling and holding of a liquid crystal display (LCD) andmay be removed by performing MCFI and increasing a refresh frame rate.The motion judder is caused by a process of converting a source framerate of a movie or the like to a frame rate of a display device and maybe removed by detecting the source frame rate, performing MCFI, and thelike.

As a content consumption pattern is expanded and changed from TVbroadcasting to streaming, the range of image quality and a format of asource is greatly diversified. In a view point of the frame rateconversion (FRC), there are contents with various frame rates of 50 Hz,30 Hz, 24 Hz, 12 Hz, 8 Hz, and the like as well as 60 Hz. In order tooutput the contents with a predetermined frame rate, for example, 60 Hz,source contents have to be converted at the predetermined frame rate.

As an example, a pattern, in which an initial frame is repeatedlyinserted between two adjacent frames of the source two times and asecond frame is repeatedly inserted therebetween, is repeatedlyperformed such that the source with a frame rate of 24 Hz is convertedat a frame rate of 60 Hz. However, since frames which are initiallydisposed with equidistant intervals in a source are not repeatedlydisposed with equidistant intervals during the conversion process,judder in which movement is not smooth occurs when the converted framesare displayed without change.

The display device detects the frame rate of the source and removesrepeatedly inserted frames to remove the judder. Then, the displaydevice performs motion estimation (ME) and motion compensation (MC)using frames, which are not repeated, to perform MCFI.

SUMMARY

In the related art, a frame rate of a source is identified by detectinga repeating pattern among input frames, and searching for a bestmatching pattern among predefined patterns. As an example, when apattern, in which any one frame is repeated three times and the nextframe is repeated two times, is repeated, a frame rate of a source isidentified as 24 Hz. In a case in which repeating patterns did not matchthe predefined patterns, a source frame rate is designated as 60 Hz.After the frame rate of the source is identified, hardware is controlledto correspond to the identified source frame rate to perform motioncompensated frame interpolation (MCFI) among a plurality of frame rateconversion scenarios.

According to a related source frame rate detection method, when a framerate in one source is mutually converted to various frame rates, anoutput frame rate is abruptly changed. In addition, when there isdigital noise or poor editing in an input frame, an abrupt change mayoccur even without a change in frame rate. In this case, since there areno matching patterns, a frame rate is changed to 60 Hz which is adefault value as a search result, judder may remain on a screen due toincorrect or rough motion estimation (ME), motion compensation (MC), andMCFI, or may rather become worse.

In addition, a frame rate conversion method according to the related artis performed by selecting a scenario, which corresponds to a detectedframe rate, among ME, MC, and MCFI scenarios predetermined according toa source frame rate. Accordingly, in the related art, since allscenarios corresponding to frame rates of existing source contents haveto be prepared, and the frame rates have to be converted, load isgenerated in terms of hardware and software.

The present invention is mainly directed to reducing disadvantages ofthe frame rate detection method and the frame rate conversion methodaccording to the related art.

A first aspect of the present invention provides a method of detecting aframe rate of a source including the steps of (a) counting the number oforiginal frames in a window having a predetermined length, whereinframes of the source are repeated and are input with an input frame rateand (b) calculating the input frame rate using a ratio between thepredetermined length and the number.

A second aspect of the present invention provides a method of detectinga frame rate of a source including the steps of (a) receiving frames ofa source provided at an input frame rate (T), (b) providing the framesto a window having a predetermined length (k) to detect the number oforiginal frames n included within the length (k), and (c) multiplyingthe input frame rate divided by the length (T/k) and the number oforiginal frames (n).

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent to those of ordinary skill in theart by describing exemplary embodiments thereof in detail with referenceto the accompanying drawings, in which:

FIG. 1 is a flowchart illustrating an outline of a method of detecting aframe rate of a source according to the present embodiment;

FIG. 2 is a view for describing an operation of the present embodimentin a case in which a source with a frame rate of 24 Hz is received at aninput frame rate of 60 Hz;

FIG. 3 is a view illustrating a process of obtaining a frame rate of asource using a window having a five frame length in a case in which asource with a frame rate of 30 Hz is provided at an input frame rate of60 Hz;

FIGS. 4 and 5 are block diagrams illustrating outlines of frame ratecalculation methods which are robust against noise;

FIG. 6 is a view illustrating an outline of an operation of the presentembodiment in a case in which frames with a source frame rate of 24 Hzare provided at an input frame rate of 60 Hz;

FIG. 7 is a flowchart illustrating an outline of a frame rate conversionmethod according to the present embodiment;

FIG. 8 is a flowchart illustrating an operation in which an output frameis formed from original frames to which input time stamps are added indetail;

FIG. 9 is a schematic view for describing an operation in which anoutput frame is formed from mother frames;

FIG. 10 is a schematic view for describing a frame rate conversionmethod according to the present embodiment;

FIG. 11 is a view for describing a process of performing motioncompensation to form a new frame;

FIGS. 12 and 13 are schematic views for describing another example ofthe frame rate conversion method according to the present embodiment,wherein a source with a frame rate of 24 Hz is up-converted at a framerate of 60 Hz;

FIGS. 14A to 14L are graphs showing results of experiments in which aframe rate of a source is detected through the frame detection methodaccording to the present embodiment and the frame rate is converted to atarget frame rate through the frame rate conversion method according tothe present embodiment; and

FIG. 15 is a graph showing a result of a case in which errors due tonoise and the like intervene into a process in which a source with aframe rate of 60 Hz is converted at a frame rate of 24 Hz.

DETAILED DESCRIPTION

However, these are provided as exemplary examples of the presentinvention, and do not limit the scope of the present invention in anyrespect. While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that there is no intent to limit theinvention to the particular forms disclosed, but on the contrary, theinvention is to cover all modifications, equivalents, and alternativesfalling within the spirit and scope of the invention.

Meanwhile, terms described in the specification should be understood asfollows.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting to the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises,”“comprising,” “includes,” and/or “including,” when used herein, specifythe presence of stated features, integers, steps, operations, elements,components, and/or groups thereof but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

Operations may be performed through an order different from thedescribed order unless the context clearly indicates a specific order.That is, the operations may be performed in the described order,substantially at the same time, or in an order opposite to the describedorder.

Sizes, heights, thicknesses, and the like of components illustrated inthe accompanying drawings referred to in described embodiments of thepresent invention are intentionally exaggerated for the sake ofconvenience in the description and ease of understanding, and are notproportionally enlarged or reduced. In addition, certain componentsillustrated in the drawings may be intentionally illustrated in anenlarged manner and the other components may be intentionallyillustrated in a reduced manner.

Unless otherwise defined, all the terms used herein have the samemeaning as commonly understood by those skilled in the art to which thisinvention belongs. It will be further understood that terms, such asthose defined in commonly used dictionaries, should be interpreted ashaving a meaning that is consistent with their meaning in the context ofthe relevant art and should not be interpreted in an idealized or overlyformal sense unless expressly so defined here.

A frame in the description of the present embodiment denotes an imageframe or progressive image frame which is de-interlaced from aninterlaced image. In addition, in the accompanying drawings, acharacter, such as A, B, or the like, added to a frame is a descriptorof the frame and is used for describing a repeating pattern of theframe.

Frame Rate Detection Method

Hereinafter, a method of detecting a frame rate of a source according tothe present embodiment will be described with reference to theaccompanying drawings. FIG. 1 is a flowchart illustrating an outline ofthe method of detecting a frame rate of a source according to thepresent embodiment. Referring to FIG. 1, the frame rate detection methodaccording to the present embodiment detects a frame rate of a source by(a) counting the number of original frames (NOF) in a window having apredetermined length when frames of a source are repeatedly input at aninput frame rate (S100), and (b) calculating a input frame rate using aratio between a length k of the window and the NOF (S200).

FIG. 2 is a view for describing an operation of the present embodimentin a case in which a source with a frame rate of 24 Hz is received at aninput frame rate of 60 Hz. Referring to FIG. 2, a frame differencesequence (FDS) is formed with respect to sequentially input frames. Inone embodiment, when a subsequent input frame is different from animmediately preceding frame, 1 is assigned thereto, and when thesubsequent input frame is the same as an immediately preceding frame, 0is assigned thereto to form a FDS.

A frame A may be expressed as 1 because of being different from animmediately preceding frame, and the next two frames may be expressed as0 because of being the same as frame A. The next frame B may beexpressed as 1 because of being different from an immediately precedingframe. The next frame may be expressed as 0 because of being the same asthe immediately preceding frame. In one embodiment, frames expressed as1 in the FDS may be stored in a buffer.

The NOF in a window w having a predetermined length k is checked (S100).Since 1 is assigned to a current frame that is different from animmediately preceding frame and 0 is assigned to a current frame that isthe same as an immediately preceding frame in the FDS, the NOF includedin the window w may be determined by counting the number of framesassigned as 1 in the window w. As illustrated in FIG. 2, the NOFincluded in a window is determined while a window w shifts.

In one embodiment, a unit in which original frames are repeated may bedetermined from the FDS. According to the embodiment illustrated in FIG.2, it can be shown that [10010] is repeated periodically, this meansthat two original frames are repeated such that one is repeated threetimes and the other is repeated two times, and a repeating unit of thefive frames is periodically repeated.

A frame rate is calculated from the detected NOF and a length of awindow. In the embodiment illustrated in FIG. 2, a window length k isfive frames, and a window w includes two original frames. In this case,a source frame rate F may be obtained by calculating an input frame rateusing a ratio between the window length k and the NOF using thefollowing Equation 1.

k:n=T:F   [Equation 1]

(k: a window length, n: an NOF, T: an input frame rate, F: a frame rateof a source)

Equation 2, in which Equation 1 is rearranged with respect to the framerate F of a source, is as follows.

$\begin{matrix}{F = {n\frac{T}{k}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

(k: a window length, n: an NOF, T: an input frame rate, F: a frame rateof a source)

That is, the frame rate F of a source may obtained by calculating theinput frame rate using a ratio between the window length and the NOF ina window using Equation 1, or by dividing the input frame rate T by thewindow length and multiplying the division result by the NOF as inEquation 2.

When calculating, the frame rate F of the source is 24 Hz, and the framerate of the source is calculated to converge to the same value while thewindow w shifts. The calculated frame rate of the source is the same asa provided frame rate of the source.

FIG. 3 is a view illustrating a process of obtaining a frame rate of asource using a window having a length of five frames in a case in whichthe source with a frame rate of 30 Hz is provided at an input frame rateof 60 Hz. Referring to FIG. 3, an FDS is formed with respect tosequentially provided frames. In the illustrated embodiment, [10]repeats in the FDS, and a repeating unit includes two frames. When arepeating unit is short, the repeating unit may be incorrectlydetermined. Accordingly, a repeating unit may be determined as fourframes in this case in order to prevent errors of the repeating unit.

An NOF n in a window w is obtained using a window having a length offive frames. In the embodiment illustrated in FIG. 3, as a window wshifts, the NOF n in the window w oscillates between 3 and 2.

When the NOF is 3, a frame rate of a source is calculated asF=3*(60/5)=36 using Equation 2. In addition, when the window shifts sothat the NOF is 2, a frame rate of the source is calculated asF=2*(60/5)=24. The calculated frame rate of the source oscillatesbetween 36 and 24 while the window w shifts. However, the frame rate iscalculated as 30 Hz when an average of the oscillating values is chosenas the frame rate, and is the same as a frame rate of the providedsource.

In one embodiment, the calculation of an average may be performed usingthe number of frames corresponding to an integer multiple of a repeatingunit of frames. As another embodiment, the calculation of an average maybe performed using the predetermined number of sequentially providedframes. The accuracy of a calculated frame rate increases when thenumber of frames for calculating an arithmetic average is an integermultiple of a repeating unit, or the number of frames increases. Inaddition, as the number of frames increases, a property of robustnessagainst noise is obtained.

FIGS. 4 and 5 are block diagrams illustrating outlines of frame ratecalculation methods which are robust against noise. Referring to FIG. 4,the NOF of counter_(i), the NOF of counter_(j), the NOF of counter_(k),the NOF of counter_(l), the NOF of counter_(m), and the NOF ofcounter_(n) in windows are obtained from FDS using the windows havingwindow lengths i, j, k, l, m, and n.

In one embodiment, a window length may be determined to correspond to aframe conversion scheme which is currently used. Table 1 shows a part ofthe currently used frame repeating schemes. Referring to the followingTable 1, currently used frame conversion schemes insert sources withframe rates of 8 Hz, 12 Hz, 24 Hz, 25 Hz, 30 Hz, 50 Hz, and the likeaccording to corresponding repeating patterns to convert to an inputframe rate (60 Hz). Repeating units of the frames according to repeatingschemes are 4, 5, 6, 10, 12, and 15 frames. In one embodiment, windowlengths i, j, k, l, m, and n for calculating the frame rates may bedetermined to correspond to the repeating units.

TABLE 1 source frame rate repeating pattern repeating unit (frames)  8Hz AAAAAAAABBBBBBB 15 12 Hz AAAAAABBBB 10 12 Hz AAAAABBBBB 10 24 HzAABBCCDDDD 10 24 Hz AABBBCCCDD 10 24 Hz AAABB 5 25 Hz AAABBCCCDDEE 12 30Hz AABB 4 50 Hz ABCDEE 6

The NOFs in windows are multiplied by values corresponding to windowlengths. When a window length is i, and an input frame rate is T Hz,NOFs are multiplied by T/i as in Equation 2. As an example, when awindow length is 4 and an input frame rate is 60 Hz, 15 (=60/4) ismultiplied by the NOF in a window to calculate a frame rate (seeEquation 2). Calculated frame rates FR_(i), FR_(j), FR_(k), FR_(l),FR_(m), and FR_(n) in windows having different lengths are added tocalculate an arithmetic average. When a frame rate FR1 is calculatedusing a plurality of windows having different lengths according to theillustrated embodiment, noise sensitivity in frame rate calculation maybe reduced because the influence of miscalculation due to noise may bereduced.

Referring to FIG. 5, time average calculations AVG_(i), AVG_(j),AVG_(k), AVG_(l), AVG_(m), and AVG_(n) are performed for the frame rateFR1 calculated at the previous stage using the plurality of lengths i,j, k, l, m, and n. A length denotes the number of the frame rates FR1used for time average calculation, and when a length is 6, a timeaverage calculation is performed for six frame rates FR1 which have beensequentially provided recently.

As an example, a length of a time average calculation may be the same asa length of a window used for counting the NOF. As another example, alength of a time average calculation may correspond to an integermultiple of a length of a window used for counting the NOF.

The time average calculation is performed for the frame rate FR1 whichis provided at the previous stage, the calculation result thereof ismultiplied by weights, w_(i), w_(j), w_(k), w_(l), w_(m), and w_(n), andthe calculation results are added to calculate a weighted average. Inone embodiment, a weight may be an integer, and when a time average ofthe frame rates FR1 converges, the weight may increase by 1 until theweight reaches a length of a time average calculation. In addition, whena time average of the frame rates FR1 does not converge, a weight maydecrease by 1 until the weight reaches 0. In one embodiment, when alength of the time average calculation is 5, a weight may be in therange of 0 to 5. A weight may increase when a calculated time averageconverges to a predetermined value and may decrease when the calculatedtime average oscillates or diverges.

FIG. 6 is a view illustrating an outline of an operation of the presentembodiment in a case in which frames with a source frame rate of 24 Hzare provided at an input frame rate of 60 Hz. The illustrated embodimentperforms a calculation using a window having lengths of 4, 5, 6, 10, 12,and 15 and an average calculator having the same lengths. Referring toFIG. 6, an FDS is obtained from sequentially provided frames. Since[10010] is repeated in the FDS, a repeating unit is 5 frames, and asillustrated in bold, it can be seen that the NOF obtained from windowshaving lengths corresponding to multiples of 5 converges.

The NOFs are multiplied by values corresponding to the lengths of thewindows and added to calculate an arithmetic average. When a windowlength is 4, and an input frame rate is 60 Hz, the NOF in a window ismultiplied by 15 (=60/4), when a window length is 5, the NOF ismultiplied by 20 (=60/5), when a window length is 6, the NOF ismultiplied by 10 (=60/6), when a window length is 10, the NOF ismultiplied by 6 (=60/10), when a window length is 12, the NOF ismultiplied by 5 (=60/12), and when a window length is 5, the NOF ismultiplied by 4 (=60/15, see Equation 2). Calculated values are added tocalculate a frame rate arithmetic average FRI.

A time average calculation is performed for the frame rate arithmeticaverage FR1 using different lengths, and calculated values, in which thecalculated results are multiplied by weights, w_(i), w_(j), w_(k),w_(l), w_(m), and w_(n), are added to calculate a weighted average.

The weight decreases by 1 when the time average diverges or oscillates,and has a minimum value of 0. In addition, the weight increases by 1when the time average converges, and has a maximum value which is alength by which the time average is calculated. When a result of thecalculation converges, the corresponding calculation result participatesin the frame rate calculation at a high weight, and values, which do notconverge, are prevented from participating in the frame rate calculationat a high weight.

As illustrated in FIG. 6, it can be seen that a frame rate FR2calculated through a second stage converges to 24 Hz which is a framerate of a source as time passes.

Frame Rate Conversion Method

Hereinafter, a frame rate conversion method according to the presentembodiment will be described with reference to the accompanyingdrawings. FIG. 7 is a flowchart illustrating an outline of a frame rateconversion method according to the present embodiment. Referring to FIG.7, the frame rate conversion method according to the present embodimentincludes detecting source frame rates of source frames provided as aninput (S150), attaching time stamps corresponding to the detected framerates to original frames among the frames provided as the input andstoring the detected frame rates with the attached time stamps (S250),forming an output frame at the time stamp which has to be output fromthe original frames to which the time stamps are attached and stored(S350).

The time stamps denote time scales at which the frames are disposed. Inone embodiment, when an interval from 0 seconds to 1 second is expressedwith the time stamp, the time stamp may be expressed with L at 0seconds, and after 1 second, the time stamp may be expressed with L at 0seconds again (a period of time stamps is L). As an example, when asource frame rate is 60 Hz, any two adjacent frames may be disposed at 0seconds and 1/60 second, and these may be denoted by the time stamps of0 and L/60.

In one embodiment, the time stamps may be common multiples of generallyused frame rates such as 8 Hz, 12 Hz, 24 Hz, 30 Hz, 50 Hz, 60 Hz, and120 Hz, and may express intervals between source frames with framerates, which are currently used or will be used in the near future, asintegers.

In another embodiment, frames may have random numbers and may not bedivided by a generally used frame rate such as 8 Hz, 12 Hz, 24 Hz, 30Hz, 50 Hz, 60 Hz, or 120 Hz. In a case in which the frames are notdivided by frame rates, time stamps at which the frames are positionedmay be obtained by performing round-up calculations.

In one embodiment, a period L of a next time stamp may be 90,000 whichis a common multiple of generally used frame rates. In anotherembodiment, the time stamp corresponding to an interval from 0 to 1second may be an integer multiple of 0 to 90,000, and the next timestamp starts from 0 again. However, this is for facilitatingmathematical calculation and not for limiting the scope of the presentinvention.

TABLE 2 a frame rate of a source time stamp interval  8 Hz 11250 12 Hz7500 24 Hz 3750 25 Hz 3600 30 Hz 3000 50 Hz 1800 60 Hz 1500 120 Hz  750

Table 2 shows time stamp intervals of frames corresponding toillustrated frame rates. As illustrated in Table 2, the frames withpredetermined frame rates are separated by time stamp intervals denotedby integers. As an example, frames with a frame rate of 60 Hz areseparated by 1/60 second. When the frames with the frame rate of 60 Hzare expressed with the time stamp interval, the frames may be expressedas being separated by the time stamp of 1500. As another example, frameswith a frame rate of 24 Hz are separated by an interval of 1/24 second.When the frames with the frame rate of 24 Hz are expressed with a timestamp interval, the frames may be expressed as being separated by thetime stamp of 3750. In one embodiment, an input time stamp attached toinput frames and an output time stamp attached to output frames whichare to be output may be calculated by the following Equation 3.

$\begin{matrix}{{{its}_{k} = {\left( {{its}_{k - 1} + \frac{L}{f_{n}}} \right){mod}\; L}}{{ots}_{l} = {\left( {{ots}_{l - 1} + \frac{L}{f_{n}}} \right){mod}\; L}}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

(f_(n): a detected frame rate of a source, f_(o): an output frame rate,L: a time stamp period, its: an input time stamp, ots: an output timestamp)

A frame rate of frames which are sequentially input is detectedaccording to above-described embodiment. To remove repeatedly providedframes, the frames denoted as 1 in an FDS are stored in a buffer with atime stamp corresponding to a detected frame rate of a source (S250).

FIG. 8 is a more detailed flowchart illustrating an operation (S350) inwhich an output frame is formed from original frames to which input timestamps are added. FIG. 9 is a schematic view for describing an operationin which an output frame is formed from mother frames F_(k) and F_(k+1).Referring to FIGS. 8 and 9, the mother frames F_(k) and F_(k+1) arechosen for forming an interpolated output frame (S352). In oneembodiment, two adjacent frames with a time stamp, which is interposedtherebetween, of output frames among frames stored in a buffer may bechosen as the mother frames F_(k) and F_(k+1). As an example, motherframes F_(k) and F_(k+1) are adjacent frames with an output time stamp,which is interposed therebetween, of frames denoted as 1 in an FDS.

A motion vector (MV) is obtained from the mother frames F_(k) andF_(k+1), and a motion compensation rate α is obtained using the timestamp (S354). The MV may be obtained by performing motion estimation(ME) on a moving object O in the mother frames F_(k) and F_(k+1). Themotion compensation ratio may be obtained using the time stamp throughthe following

Equation 4.

$\begin{matrix}{\alpha = {\frac{{td}\; 1}{{{td}\; 1} + {{td}\; 2}} = \frac{{ots}_{l} - {its}_{k}}{{its}_{k + 1} - {its}_{k}}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

An interpolated frame (F_(MC)) is formed at an output time stamp usingan MV and a motion compensation rate obtained by performing ME andmotion compensation (S356). In one embodiment, an output time stamp of amother frame and an output time stamp of an interpolated frame may bethe same. As an example, in a case in which a source frame rate is 30Hz, and an output frame rate is 60 Hz, an output time stamp of a motherframe and an output time stamp of an interpolated frame may be the same,and in this case, the mother frame may be output at the output timestamp. As another example, a motion compensation rate α may be 0, thiscase is also the same as the case in which an output time stamp of amother frame and an output time stamp of an output frame are the same,and in this case, the mother frame may also be output at the output timestamp.

Hereinafter, an example of the frame rate conversion method according tothe present embodiment will be described with reference to theaccompanying drawings. FIG. 10 is a schematic view for describing aframe rate conversion method according to the present embodiment. In anexample illustrated in FIG. 10, a source with a frame rate of 30 Hz isup-converted at a frame rate of 60 Hz. Here, the source is stored withan input time stamp, an output is formed at an output time stamp, buthereinafter, a source and an output are disposed at the same time stampfor the sake of simple description and ease of understanding.

Referring to FIG. 10, since adjacent source frames of 30 Hz areseparated by a time stamp interval of 3000, and an output has a framerate of 60 Hz, adjacent output frames have to be separated by a timestamp interval of 1500. In addition, the output frames may be formedwith a corresponding output frame rate by forming adjacent mother framesF1 and F2, which are illustrated with solid lines, of a source and oneF_(MC) obtained by performing motion compensated frame interpolation onthe adjacent input frames at a target time stamp of 1500.

FIG. 11 is a view for describing a process of performing motioncompensation to form a new frame. Referring to FIG. 11, it is assumedthat an object O moves in two adjacent mother frames F1 and F2. Themother frames F1 and F2 for forming an output F_(MC) are chosen (S352).As described, adjacent frames with a time stamp of 1500, which isinterposed therebetween, and at which an output F_(MC) is formed, amongframes stored in a buffer may be chosen as the mother frames F1 and F2.

An MV and a motion compensation rate α are obtained from the motherframes F1 and F2 (S354). Even in a case in which the object Oaccelerates, the object O may be approximated to move at a constantspeed for a short time period corresponding to an interval betweenframes.

From the approximation, ME is performed on the moving object to obtainthe MV, and MC is performed to obtain a motion compensation rate α. Themotion compensation rate α may be calculated using a time stamp asfollows.

$\begin{matrix}{\alpha = {\frac{{td}\; 1}{{{td}\; 1} + {{td}\; 2}} = {\frac{{ots}_{l} - {its}_{k}}{{its}_{k + 1} - {its}_{k}} = {\frac{1500 - 0}{3000 - 0} = 0.5}}}} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

An F_(MC) is formed using a compensated MV αMV by the motioncompensation rate α, and the F_(MC) is disposed at a time stamp of 1500corresponding to an output frame rate to perform frame rate conversion(S356). In addition, since output time stamps of the mother frames F1and F2 are the same as input time stamps, the mother frames F1 and F2may be used as output frames.

FIGS. 12 and 13 are schematic views for describing another example ofthe frame rate conversion method according to the present embodiment,wherein a source with a frame rate of 24 Hz is up-converted at a framerate of 60 Hz. FIGS. 12 and 13 are schematic views for describing aninterpolated frame formed at a time stamp of 3000 using mother frames F1and F2. Referring to FIGS. 12 and 13, adjacent frames with the timestamp of 3000, which is interposed therebetween and corresponds to anoutput frame rate, are chosen as the mother frames F1 and F2 (S352). Asdescribed, the adjacent frames with the time stamp of 3000, which isinterposed therebetween and at which an output F_(MC) is formed, amongframes stored in a buffer may be chosen as the mother frames F1 and F2.

In one embodiment, new frames are formed from the adjacent mother framesF1 and F2 by performing MC, but any one of the mother frames may not beincluded in an output and may be removed. Since an input time stamp andan output time stamp of the mother frame F1 are the same (or, a motioncompensation rate is 0), the mother frame F1 is output as an outputframe.

ME is performed on a moving object O in the two adjacent source framesF1 and F2 to obtain an MV, and MC is performed to obtain a motioncompensation rate α (S352). Accordingly, the motion compensation rate αfor the object O may be calculated using time stamps td1 and td2 throughEquation 3. As an example, a motion compensation rate α is calculated atan F_(MC2), which will be newly formed, using the following Equation 6.

$\begin{matrix}{\alpha = {\frac{{td}\; 1}{{{td}\; 1} + {{td}\; 2}} = {\frac{{ots}_{l} - {its}_{k}}{{its}_{k + 1} - {its}_{k}} = {\frac{3000 - 0}{3750 - 0} = 0.8}}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

An F_(MC) may be formed using a compensated MV αMV by the motioncompensation rate a obtained using the time stamps, and the F_(MC) maybe disposed at the time stamp of 3000 corresponding to an output framerate to perform frame rate conversion (S356).

Experiment Results

FIGS. 14A to 14L are graphs showing results of experiments in whichframe rates of sources are detected through the frame detection methodaccording to the present embodiment, and the frame rates are convertedto target frame rates through the frame rate conversion method accordingto the present embodiment.

FIG. 14A is a view illustrating a case in which a source with a framerate of 60 Hz is converted at a frame rate of 8 Hz, 12 Hz, 24 Hz, 25 Hz,30 Hz, or 50 Hz, and FIG. 14B is a view illustrating a case in which asource with a frame rate of 8 Hz, 12 Hz, 24 Hz, 25 Hz, 30 Hz, or 50 Hzis converted at a frame rate of 60 Hz. FIG. 14C is a view illustrating acase in which a source with a frame rate of 24 Hz is converted at aframe rate of 8 Hz, 12 Hz, 25 Hz, 30 Hz, 50 Hz, or 60 Hz, and FIG. 14Dis a view illustrating a case opposite the case of FIG. 14C. FIG. 14E isa view illustrating a case in which a source with a frame rate of 30 Hzis converted at a frame rate of 8 Hz, 12 Hz, 25 Hz, 24 Hz, 50 Hz, or 60Hz, FIG. 14F is a view illustrating a case opposite the case of FIG.14E. FIG. 14G is a view illustrating a case in which a source with aframe rate of 12 Hz is converted at a frame rate of 8 Hz, 30 Hz, 25 Hz,24 Hz, 50 Hz, or 60 Hz, and FIG. 14H is a view illustrating a caseopposite the case of FIG. 14G. FIG. 14I is a view illustrating a case inwhich a source with a frame rate of 8 Hz is converted at a frame rate of12 Hz, 30 Hz, 25 Hz, 24 Hz, 50 Hz, or 60 Hz, and FIG. 14J is a viewillustrating a case opposite the case of FIG. 14I. FIG. 14K is a viewillustrating a case in which a source with a frame rate of 50 Hz isconverted at a frame rate of 12 Hz, 30 Hz, 25 Hz, 24 Hz, 8 Hz, or 60 Hz,and FIG. 14L is a view illustrating a case opposite the case of FIG.14K.

Referring to FIG. 14, it can be seen that in a process in which a sourcewith any one frame rate is converted at another frame rate, the framerate conversion is rapidly completed because the conversion is completedwithin about 12 frames to 22 frames. In addition, even in a case inwhich a frame rate is abruptly changed, glitches do not occur so thatthe frame rate conversion is smoothly completed. From such a result,there are advantages in that more natural and smooth ME and MC can beprovided.

FIG. 15 is a graph showing a result of a case in which errors due tonoise and the like intervene into a process in which a source with aframe rate of 60 Hz is converted at a frame rate of 24 Hz. Referring toFIG. 15, even in a case in which a plurality of errors intervene intothe process, it can be seen that frame rate conversion is relativelyrapidly completed when compared to a case in which there are no errors(0 errors). Accordingly, it can be seen that the present embodiment hasa property of robustness against noise.

As described above, there are advantages in that a frame rate detectionmethod according to the present embodiment is robust against noise andincorrect editing, and can detect a correct frame rate. In addition,there are advantages in that resources consumed to perform the framerate conversion method can be reduced and a smooth conversion in framerate can be performed even when abrupt frame rate conversion isperformed because the frame rate conversion method according to thepresent embodiment based on a time stamp is not a scenario-based method.

While the invention has been described with reference to the embodimentsillustrated in the accompanying drawings for promoting an understandingof the present invention, the embodiments should be considered in adescriptive sense only, and it should be understood that variousalterations and equivalent other embodiments may be made by thoseskilled in the art. Therefore, the scope of the invention is defined bythe appended claims.

1-8. (canceled)
 9. A method of detecting a frame rate of a source,comprising the steps of: (a) receiving frames of a source provided at aninput frame rate (T); (b) providing the frames to a window having apredetermined length (k) to detect the number of original frames (n)included within the window having the length (k); and (c) multiplyingthe input frame rate divided by the length (T/k) and the number oforiginal frames (n).
 10. The method of claim 9, further comprisingcalculating an arithmetic average of results of the step (c).
 11. Themethod of claim 9, wherein the step (b) is simultaneously performed witha plurality of windows having different lengths.
 12. The method of claim11, further comprising (d) calculating an arithmetic average of resultsof the simultaneous performance.
 13. The method of claim 12, furthercomprising: (e) providing a result of the step (d) to a plurality oftime average calculators having different lengths to calculate timeaverages corresponding to the lengths; and (f) calculating a weightedaverage of results calculated in the step (e).
 14. The method of claim13, wherein, in the step (f), a weight of the weighted average increasesas the average calculated in the step (e) converges.
 15. The method ofclaim 13, wherein, in the step (f), a weight of the weighted averagedecreases as the average calculated in the step (e) does not converge.16-33. (canceled)